-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bigtable: fix handling of unset rpc timeouts #4323
Merged
igorbernstein2
merged 1 commit into
googleapis:master
from
igorbernstein2:zero-disable-timeout
Jan 11, 2019
Merged
Bigtable: fix handling of unset rpc timeouts #4323
igorbernstein2
merged 1 commit into
googleapis:master
from
igorbernstein2:zero-disable-timeout
Jan 11, 2019
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When the rpc timeout is zero, then it should be treated as disabled not actual 0
igorbernstein2
added
type: bug
Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
api: bigtable
Issues related to the Bigtable API.
labels
Jan 10, 2019
googlebot
added
the
cla: yes
This human has signed the Contributor License Agreement.
label
Jan 10, 2019
igorbernstein2
added
the
kokoro:force-run
Add this label to force Kokoro to re-run the tests.
label
Jan 10, 2019
yoshi-kokoro
removed
the
kokoro:force-run
Add this label to force Kokoro to re-run the tests.
label
Jan 10, 2019
sduskis
approved these changes
Jan 11, 2019
snehashah16
added a commit
that referenced
this pull request
Feb 3, 2019
* Fix: writeAtLeastOnce should use a ReadWriteSession (#3882) * Fix search folder for BQDT synth output (#3878) * Bigtable: Add integration test to test nonexistent row handling (#3870) * Fix #3292 (#3875) * Regenerate video-intelligence client (#3865) * Release v0.69.0 (#3890) * Bump next snapshot (#3892) * Bigtable: fix admin tests to able to run concurrently (#3895) * Fix ApiFutures.addCallback() (#3886) ApiFutures.addCallback() now requires a third argument (a non-direct executor). Ref: https://github.com/googleapis/api-common-java/blob/master/src/main/java/com/google/api/core/ApiFutures.java * Regenerate logging client with test fixes (#3907) * Regenerate DLP Client (#3823) * Regenerate vision client: Adds product search (#3888) * Regenerate vision client * vision proto libraries need com.google.api:api-common compile dependency * Regenerate dataproc client (#3906) Unsets empty list of retry status codes * Regenerate monitoring client (#3908) Unsets empty list of no-retry status codes * Move BOM info to right after Quick Start (#3555) * Move BOM info to right after Quick Start * split version management * change to old version combinations * Update README.md * Spanner gapic migration (#3881) * Add GapicSpannerRpc stub (#3016) * spanner: implement partitionRead (#3022) * spanner: implement LRO methods using GAPIC stub (#3039) * Add implementation to:listInstances, listInstanceConfigs, listDatabases, getInstance, getInstanceConfigs, getDatabase, dropDatabase, deleteInstance, getDatabaseDdl (#3043) * spanner: move admin clients to GAPIC stub (#3067) * Spanner: migrate all unary call methods to gapic and inject headers (#3112) * Spanner: migrate streaming methods to gapic (#3139) * Spanner gapic migration lro methods (#3166) Migrate longrunning methods to GAPIC including: createDatabase, updateDatabaseDdl, createInstance, and updateInstance. * Make ChannelPool work (#3258) * Spanner gapic migration error augmentation with interceptor (#3304) * Add interceptors (#3346) * Clean up Spanner before merging to master (#3362) * Spanner Gapic Migration: fix updateDatabaseDdl (#3403) * Regenerate compute client (#3916) Adds interconnect features * Regenerate dlp client (#3917) Add excluded_fields to BigQueryOptions * Add a feature to list the buckets (#3915) Also, update the README to reflect recent update to pseudoDirs. * Pubsub: use gapic client (#3581) * bump checkstyle version to build on Java 9 The version of checkstyle we currently use uses tools.jar which is removed from Java 9 and above as part of Project Jigsaw. This commit uses a newer version of checkstyle that does not use tools.jar. Running `mvn checkstyle:checkstyle` succeeded.� * maybe this will be java 7 compat...? * pubsub: use GAPIC stub * fix merge * pr comment * Update CODEOWNERS fallback match should be last * Storage NIO: Stop leaving leftover buckets in IT (#3898) * Stop leaving leftover buckets in ITGcsNIO This integration test now deletes all the buckets it creates. The issue was that requester-pays buckets couldn't be deleted by the existing code. This PR updates RemoteStorageHelper so it can be given a userProject for requester-pays buckets. * Don't set userProject if empty, add test RemoteStorageHelper now doesn't set the userProject option if it's empty. Also, added testForceDeleteRetriesWithUserProject unit test. * Logging: Add a traceSampled field to the LogEntry class. (#3863) The new field was added to the Stackdriver LogEntry protocol buffer message in googleapis/googleapis@7b34e5a#diff-a51916a680df6453b6d86586cd7d63f9R149. * Bigtable: add sync methods (#3856) * New Client: Cloud Scheduler (#3913) * Add synth.py for generating cloud scheduler client * Add generated cloud scheduler code * Fix proto/grpc artifact names * Add versions and pom.xml files * Update list of stub packages in google-cloud-clients * Add README * Exists method added in BigtableInstanceAdminClient (#3891) * Firestore: Validate that at least one argument is provided for Cursor (#3900) * Release google-cloud-java v0.70.0 (#3921) * Bump next snapshot (#3922) * bigtable emulator (#3840) This allows customers to easily interact with the bigtable emulator. Currently the Bigtable emulator is implemented in go. It is a fairly complex bit of code that would be very difficult to replicate in java. This makes it very difficult for users to interact with it in a java project. This PR attempts to make things easier for the user by wrapping the golang emulator in a jar and providing a JUnit rule to simplify lifecycle management of the emulator. It's implemented as 2 artifacts: - google-cloud-gcloud-maven-plugin: a maven plugin that can download artifacts from the gcloud sdk repo and add them as resources to a jar. This avoids the need to check in binaries into google-cloud-java and makes sure that the binaries are up to date. - google-cloud-bigtable-emulator: This uses the maven plugin to bundle the bigtable emulator in a jar and adds a java wrapper around the binary. The wrapper was extracted from the existing bigtable client integration tests: https://github.com/googleapis/google-cloud-java/blob/master/google-cloud-clients/google-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/it/env/Emulator.java * Update README to suggest importing BOM (#3269) * Import BOM @garrettjonesgoogle * google-cloud-bom * Update README.md * Spanner: use WatchdogProvider (#3923) * change to watchdog provider * Bigtable: [Breaking change] listTables() now returns relative table names (#3696) * Bigtable: be consistent and don't expose absolute table names * fix merge * fix merge * Update container synth script for new artman config name (#3925) * New Client: BigQuery Storage (#3935) * New Client: Cloud Security Command Center (#3885) * Add synth.py script and generate files * Add versions for release management, add README.md, pom.xml * Add google-cloud-security center as module for google-cloud-clients * Add securitycenter to google-api-grpc modules * Add securitycenter artifacts to the bom * security center depends on iam protos * Regenerate Security Center models/stubs * Fix artifact versions * Fix artifact versions * Fix artifact versions * Add Kokoro nightly build configs (#3926) * Add namespace and schema to settings.xml (#3941) Updates the release setup guide. * Regenerate bigquerystorage client (#3939) Adds documentation strings * fix: google-cloud-clients/pom.xml to reduce vulnerabilities (#3943) The following vulnerabilities are fixed with an upgrade: - https://snyk.io/vuln/SNYK-JAVA-COMMONSFILEUPLOAD-30401 * Regenerate speech client (#3933) Adds `result_end_time` to `StreamingRecognitionResult` * Update google-auth-library-java, apiary core libraries (#3945) * Update google-auth-library-java, apiary core libraries * Fix static import from shaded guava package * Upgrade gax, grpc and guava dependencies (#3980) * Bump gax version to 1.35.0 and guava to 26.0-android * Update grpc dependency to 1.16.1 * Add temporary, specific animal-sniffer-annotation version * Release v0.71.0 (#3978) * Bump next snapshot (#3981) * Fix BigQuery NullPointerException when estimatedFields is empty (#3984) * BigQuery client throws NullPointerException when estimatedFields is empty * fix formatting * Regenerate container client (#3979) * Cloud Asset fix README links (#3985) Updated Cloud Asset links to point to the Cloud Asset API page. * Add synth metadata for many clients (#4020) * Storage: Added feature to change host name of signed url (#3910) * #3140 feature added to change hostname of generated Url * add test case * removed overload method and add method withExtHostName in SignUrlOption * rename EX_HOST to SERVICE_ENDPOINT,rename withExtHostName to withHostName and used google-java-formatter * update as per feedback * fix formatting * update comments * update Signed URLs Comment * Regenerate websecurityscanner client (#4080) * Delay download of Bigtable emulator (#4102) * Add gcloud.download.skip option for google-cloud-gcloud-maven-plugin * Skip emulator download when doing the initial install * Rename `skip` to `shouldSkipDownload` * [Snyk] Fix for 1 vulnerable dependencies (kr.motd.maven:os-maven-plugin) (#4097) * fix: google-cloud-util/google-cloud-compat-checker/pom.xml to reduce vulnerabilities The following vulnerabilities are fixed with an upgrade: - https://snyk.io/vuln/SNYK-JAVA-ORGCODEHAUSPLEXUS-31521 * Add guava dependency in google-cloud-compat-checker Previously, the kr.motd.maven:os-maven-plugin brought in a dependency that brought in guava. This artifact explicitly uses guava collections so we need to add the dependency here. * fix: google-cloud-testing/google-cloud-gcloud-maven-plugin/pom.xml to reduce vulnerabilities (#4100) The following vulnerabilities are fixed with an upgrade: - https://snyk.io/vuln/SNYK-JAVA-ORGCODEHAUSPLEXUS-31521 * Trace: list response returns empty list instead of null (#4079) * Regenerate speech client: remove unused imports (#4077) * Update generated code: List responses return empty list instead of null (#4104) * Regenerate automl client * Regenerate bigquerydatatransfer client * Regenerate bigtable-admin client * Regenerate compute client * Regenerate containeranalysis client * Regenerate dataproc client * Regenerate dialogflow client * Regenerate dlp client * Regenerate errorreporting client * Regenerate firestore client * Regenerate kms client * Regenerate logging client * Regenerate redis client * Regenerate scheduler client * Regenerate securitycenter client * Regenerate spanner client * Regenerate tasks client * Regenerate language client: remove unused imports (#4071) * Datastore: Add `skippedResults` in `QueryResults` (#4093) * Fix #3279 google-cloud-datastore Java library exposes too little of the information in query results * fix typo in comment * BigQuery Storage: Add shim layer settings and configuration (#4022) * Create a shim layer for the BigQuery Storage API. This change adds a simple shim layer to the BigQuery Storage API client. It is modeled after the shim layer in the Cloud Bigtable client, but unlike that implementation, it does not create chained instances of unary and server-streaming callables to perform translation; instead, its parameters are simple pass-throughs to the underlying gRPC client stub. The shim currently does nothing but read default values from the base client layer and pass them back through. * Set the default gRPC inbound message size. This change sets the default value for the maximum inbound message size in the BigQuery Storage API client. A ReadRowsResponse object can contain serialized row block of up to 10 MB in size if the underlying row(s) are sufficiently large. * Update the default timeout value for ReadRows. This change modifies the default client-side timeout value for the ReadRows API. ReadRows calls can last for long periods in the case of large tables. The default timeout value is set to a full day, which is the duration for which a read session is valid. * Add a resumption strategy for ReadRows. This change modifies the configuration for the ReadRows API to add a resumption strategy, allowing the connection to be resumed transparently in the case of transient errors. * Spanner: fix code formatting (#4105) * Storage NIO: Add new retry options in CloudStorageConfiguration (#3869) * Put new retry options in CloudStorageConfiguration Add retriableHttpCodes and reopenableExceptions so the user can, if they choose, customize the conditions under which we retry an access or reopen a channel after an exception closed it. This can be handy if a cloud provider starts transiently throwing exceptions that would normally indicate a fatal condition, but which in practice will pass if retried. The backoff logic in retries / reopens in unmodified. * Clean up dead code line * retriable -> retryable (in NIO Cloud Storage code) Also add a few basic tests to make sure the retryable options do what we expect. * Update testListFilesInRootDirectory to match reality In the built-in UNIX filesystem, the newDirectoryStream method will return absolute paths if given absolute paths as input, and relative paths if given relative paths as input. We're now seeing this too for the NIO Cloud Storage provider (this is a good thing), so I updated this test to reflect this new reality. * remove unused import * Add boilerplate file header * Keep old ctor, just @deprecated * Add back the *other* ctor that needs to be deprecated * add @deprecated outside of comment * Regenerate IoT client (#4069) Adds new API endpoints for: * sendCommandToDevice * bindDeviceToGateway * unbindDeviceFromGateway * Moving surefire-junit4 out of the dependency lists to allow others to better run tests. (#4058) * Try adding maven-surefire-plugin as a plugin to pull the correct surefire-junit. * Let's move the surefire-junit4 out of util as well to be consistant. * Fix indentation in pom * Regenerate dataproc client - Adds workflow templating (#4110) * Add missing fields on MonitoredResourceUtil (#3887) * added missing fields #3559 * Fix indentation * Add helpers to convert core Date class with java.util.Date class (#3911) * #3157 add methods toDate, fromDate * resolved test case Date.getYear() * update as per feedback * Update javadoc string * Bigtable: publish emulator to maven central (#4106) * Bigtable: publish emulator to maven central * make sure nexus plugin is not disabled * Release v0.72.0 (#4113) * Adding FieldMask support to GetAll() (#4017) * Bump next snapshot (#4118) * Logging: Allow deferred formatting for trace and spanId (#3661) * Allow deferred formating for trace and spanId. * Add back setTrace(String) and setSpanId(String) for binary compatibility * Update HttpStorageRpc.java (#4109) * Make TestServiceFactory and TestServiceRpcFactory interfaces public (#4112) * Java 11 compatibility (#3821) * Add the javax.annotation-api as a compile scope to the google-api-grpc projects * Upgrade maven-compiler-plugin 3.5.1->3.7.0 and maven-jar-plugin 2.6->3.1.0 * Add annotation-api compile dependency to google-cloud-clients * Fix objensis version 3.1.0->2.6 * Fix service option test visibility * Temporarily using Java 7 build to test Java 11 * Restore the java 7 test to use the java 7 image * Add release profile to bigtable emulator (#4119) * Release v0.72.0 * Add release profile to bigtable emulator * Add release profile to bigtable emulator * Add lint test and format generated code (#4114) * Add lint Kokoro tests * synthtool will now run the java formatter after generating the files * BigQuery: Fix TableId behaviour for non default BigQueryClient project (#4115) * Fixes create, update and delete methods * remove comments * Fix formatting * Run google-java-format on all artifacts (#4121) * Run google-java-format on all artifats * Format some missed files * Add more missed files * Fix formatting issues from the merge * Pub/Sub: Update ApiFutures.addCallback() to take 3 arguments (#4120) * Update ApiFutures.addCallback() * use DirectExecutor * nit * use directExecutor * run formatter * Update maven-javadoc-plugin to 3.0.1 and disable doclint. (#4167) 3.0.0 introduced a setting to disable doclint via config instead of a necessitating a conditional profile based on the java version. * Remove the old clients compatibility chart. (#4163) Any of these clients in question would be using versions older than a year and should upgrade. * BigQuery: insertAll always retries operation (#4165) * fix #3321 BigQuery insertAll always retries operation * Update Spanner description for GA (#4174) * Storage: Close response body after request (#4116) * 4107: Disconnect response after request * 4107: Added integration test testWriteChannelWithConnectionPool * 4107: Renamed blobName. * Fix lint * Storage: Fixed link to StorageExamples page (#4173) * 4169: Fixed link to StorageExamples page * 4169: Fixed Line 39 An App Engine application that manages a virtual bookshelf. * 4169: Removed WS * Integration test deflake (#4176) * Bump integration test timeout to 15 minutes from 10 minutes * Call the maven test directly - no need for verify_single_it.sh * provide spanner.testenv.instance property from Kokoro config * Bump integration test timeout to 20 minutes * Fix syntax * Remove exclusions for guava-jdk5 and google-api-client (#4177) We've updated the apiary clients to versions that no longer need these exclusions. * Storage: Give users the ability to disable gzip content encoding to increase throughput (#4170) * 3822: Added BlobTargetOption for user to set disabledGzipContent true * 3822: Changed method comment * 3822: Changed method comment with more explanation * 3822: Fixed comments after review. Added unit test * 3822: Fixed formatting errors in Storage class * 3822: Fixed formatting errors in StorageImplTest class * Spanner: Handle AbortedExceptions that occur before commit (#4181) * spanner: Handle AbortedExceptions that occur before commit The current TransactionRunner code checks for aborted transactions by using the isAborted() function. This function returns the value of "aborted" that is only set when processing resultSets. This does not work for DML which only uses the resultSet stats and does not actually call ResultSet.next(). This patch adds an additional check for the aborted exception type and retries such transactions. * throw before executeUpdate, verify flag is false after test * Update google-cloud-clients/google-cloud-spanner/src/test/java/com/google/cloud/spanner/it/ITDMLTest.java Co-Authored-By: nithinsujir <118742+nithinsujir@users.noreply.github.com> * Put the spanner test instance config back * Document purpose of test-jar and testlib dependencies (#4164) * google-cloud-bom should not need google-cloud-core test-jar (#4168) * google-cloud-bom should not need google-cloud-core test-jar * Need to declare google-cloud-core version still * testing modules need to specify the google-cloud-core test-jar version * Regenerate pubsub client: Fix empty list returns (#4159) * Regenerate pubsub client * Fix the default maxInboundMessageSize config via synth * BigQuery: Allow job to extract or load table across projects (#4183) * Fix #3924 ExtractJobConfiguration's setProjectId makes cross-project BQ extracts impossible * Bigtable: fix emulator publishing (#4189) * Bigtable: fix emulator publishing Currently the emulator can't be used because it's parent pom is not published. THis PR removes the parent and copies the relevant bits into the emulator's pom Fixes #4187 * Add a couple more bits from the parent pom * oops * Release google-cloud-java v0.73.0 (#4191) * Release v0.73.0 * Fix version replacements * Move Spanner to GA section * Fix doclint option * Fix pom issues for Bigtable emulator * Bump next snapshot (#4192) * Add Container synth.metadata (#4193) * Add Pub/Sub synth.metadata (#4194) * Update variable naming. (#4199) * Update api-common-java javadoc link. (#4200) * Regenerate compute client: adding beta config classes/options (#4201) * Regenerate dataproc client: Adding beta APIs for workflow templates (#4202) * NIO: Enabled blocked tests. Fixed testCantCreateWithoutUserProject test. (#4182) * 3448: Ublocked tests. Fixed testCantCreateWithoutUserProject test. * 3448: Fixed code format. * 3448: Fixed code format issue * Bigtable: readRow methods added with Query as argument (#4092) * Regenerate automl client - No substantive changes (#4209) * Regenerate compute client (#4210) This PR was generated using Autosynth. 🌈 Adding new resource types and methods from updated Discovery doc. * Fix setProjectId for insertAll (#4196) * Pubsub: Test for blocking push subscriptions for vpc-sc enabled projects (#4214) * Bump gax version to 1.35.1 (#4217) * Add IamCredentials client (#4216) * Add IamCredentials client * Fix pom versions and formatting * Remove classes that were in my cache directory from a bad previous generation * Remove some more files cached from a bad synth * Fix xml formatting * Fix xml formatting * Update README descriptions and links * Fix generated code lint * Synth script should format java code * Release v0.74.0 (#4222) * Bump next snapshot (#4224) * BigQuery : Fix setProjectId (#4227) * Fix Change in TableId behaviour for non default BigQueryClient project * Fix setProjectId for TableInfo * add test cases and fix writechannel * fix format code * Add BadRecords to LoadStatistics (#4230) * Add script to help bootstrap new client libraries (#4111) * Add new client python script * Add requirements.txt, convert to use attrs * Start new artifact versions at 0.0.0-alpha. Next release will bump version * Switch to pathlib * Move main execution into main() * Standardize quotes to double-quotes * Update docstrings * Fix Path vs. string types when reading/writing files * Update synth.py template to include the java formatter * Update javadoc link for api-common-java. (#4236) * IAMCredentials: Fix name/description in pom (#4238) * Regenerate google-cloud-compute (#4213) Generation of Compute client with latest gapic-generator changes. This also contains one set of manual changes in ITComputeTest, where we remove the hack needed to parse a ResourceName string. * Ignore testRotateFromCustomerEncryptionToKmsKeyWithCustomerEncryption while we work out backend change (#4266) * Fix NPE in BQ's StandardTableDefinition (#4247) * Bigtable: add a separate callable for point reads (#4264) * Spanner: closed watch-dog daemon threads spawned by Spanner client (#4263) * Fixed Issue - shutting-down WatchDog daemon threads spawned by Spanner client * Fixed Issue - shutdown watchDog daemon threads spawned by Spanner client * Minor code formatting * Fixed Kokoro code format issue * Fix Kokoro release jobs (#4225) * Fix Kokoro release jobs. Needs loopback pinentry-mode * Skip install on deploy * skip stagingProfileId * Capture built artifacts * Enable promote/drop jobs via environment variable * Fix directory path and use -B for drop/release * promote/drop need the settings file * Fix path to settings file * Update RELEASING documentation to use the Kokoro jobs * Pub/Sub: Add message abandonment (#4250) * add abandonment * add unit test for abandon * update forget() to stop extending deadline * BigQuery: Fix Maximum Bytes Billed property. (#4262) * Fix Maximum Bytes Billed * remove whitespace * Update UsePubSubEmulatorSnippet.java (#4256) * Logging: disable severity based flush on write by default (#4254) * fix #3880 #2796 disable automatic flush on write to avoid hanging and leaking threads as grpc also uses this appender which goes in to deadlock over time. * fix formatting * add flush before listing * Docs: Fix maven site generation (#4232) The configuration for aggregate changed from an attribute to a report * Bigtable: deprecate the use of typesafe names (#4257) * Bigtable: deprecate the use of typesafe names Unfortunately typesafe names create more issues for bigtable than they solve. A lot of the time users need to use both bigtable data & admin apis. Unfortunately, those apis define the same names in different namespaces. This forces users to use fully qualified names which is ugly. Also, absolute names don't benefit the bigtable client because the client is anchored at the instance and all names end up being relative * fix docs * format code * Fixed cannot call public methods in via reflection (#4249) * modified package-private to public * update format for storage/Option.java * BigQuery : Fix numBytes, numRows in TableInfo/Table (#4271) * Fix NumBytes and NumRows * update NumBytes and NumRows * remove setter * update methods and resolved test case * Cloud Bigtable: helloWorld sample (#4274) * helloWorld sample * test class modified * format change in ITHelloWorld * Add .tmpl extension for README.md so releasetool is not confused (#4280) * Release v0.75.0 (#4282) * Bump next snapshot (#4286) * Fix #4284 NPE for getZone (#4289) * Regenerate securitycenter client (#4290) * Bigtable admin: deprecate typesafe names part 2 (#4258) * Bigtable admin: deprecate typesafe names This is a followup on #4257: Unfortunately typesafe names create more issues for bigtable than they solve. A lot of the time users need to use both bigtable data & admin apis. Unfortunately, those apis define the same names in different namespaces. This forces users to use fully qualified names which is ugly. Also, absolute names don't benefit the bigtable client because the client is anchored at the instance and all names end up being relative * format code * fix year * Regenerate iamcredentials client (#4277) * Revert "Pub/Sub: Add message abandonment (#4250)" (#4297) This reverts commit 6e2c2dd. * updated links (#4287) * updated links * updated link * added new line * Release v0.76.0 (#4298) * Regenerate dataproc client (#4301) * Bump next snapshot (#4300) * Cloud Bigtable: instanceAdmin sample (#4299) * instanceAdmin sample and tests * comments added * instanceAdminExample changes * renamed variables * changes * Regenerate securitycenter client (#4311) * 3540: Used transformer for shaded plugin (#4305) * Added Query#fromProto to convert protobuf ReadRowsRequest (#4291) * Adding Query.fromProto * Adding Query.fromProto * adding changes as per feedback * updated javadoc as per feedback * reformatted Query & QueryTest.java * Bigquery: corrected equality check on subclasses of StringEnumValue (#4283) * Bigquery: corrected equality check on subclasses of StringEnumValue * update format for FieldTest.java * Fix #4269 update metata url to FQDN (#4278) * 4152: Added checking PAGE_TOKEN from options (#4303) * 3918: Added checking of attributes size for 0. Added unit test (#4304) * Bigtable: Merge bigtable-admin into the bigtable artifact (#4307) * Bigtable: merge bigtable-admin into the bigtable artifact * split exclusion change for future PR * fix admin pathes * fix deprecation warning * fix synth script * fix generated file * temporarily re-add kokoro scripts (and point them to the merged artifact) until the jobs have been removed * Remove dep from examples * fix admin integration tests * revert stray fix (will be added to a separate PR) * fix int tests * don't double format the code * fix up docs * Regenerate PubSub: documentation updates (#4293) * Regenerate monitoring client (#4316) * Regenerate bigtable client (#4318) * 3822: Fixed setDisableGZipContent call (#4314) * 3822: Fixed setDisableGZipContent call * 3822: Changes after review * Regenerate speech client: enable multi-channel features (#4317) * Release v0.77.0 (#4324) * Created enum Region.java to retrieve Regions without doing request. (#4309) * Bump next snapshot (#4325) * Bigtable: fix handling of unset rpc timeouts (#4323) When the rpc timeout is zero, then it should be treated as disabled not actual 0 * Bigtable: Expose single row read settings (#4321) * exposing ReadRowSettings thru BigtableDataSettings * fixed typo error * Regenerate compute client (#4327) * Regenerate speech client (#4328) * Update README version to use bom version (#4322) * BigQuery: Fix NPE for null table schema fields (#4338) * Add failing test for empty table schema * Fix NPE if table schema returns null for fields * Bump gax, grpc & opencensus version (#4336) * Cloud Bigtable: HelloWorld sample updates (#4339) * comments added in HelloWorld and ITHelloWorld * removed typsafe name * separate properties for bigtable.project and bigtable.instance * separate properties for bigtable.project and bigtable.instance (#4346) * Pub/Sub: default values in batch settings comments (#4350) * Pub/Sub: default values in batch settings comments Ref: https://github.com/googleapis/gax-java/blob/master/gax/src/main/java/com/google/api/gax/batching/BatchingSettings.java * Verbose comment * Update maven-surefire-plugin to 3.0.0-M3 to fix Java 8 classloader (#4344) * Update maven-surefire-plugin to 3.0.0-M3 to fix Java 8 classloader * Update failsafe plugin to 3.0.0-M3 too * Specify maven-surefire-plugin version in bigtable-emulator * Bigtable: Fixing a typo for KeyOffSet#geyKey to getKey (#4342) * Regenerate spanner client (#4332) * Spanner: remove junit from compile dependencies (#4334) * Firestore: change timestampsInSnapshots default to true. (#4353) BREAKING CHANGE: The `areTimestampsInSnapshotsEnabled()` setting is now enabled by default so timestamp fields read from a `DocumentSnapshot` will be returned as `Timestamp` objects instead of `Date`. Any code expecting to receive a `Date` object must be updated. * Regenerate clients with updated copyright year (#4382) * Regenerate asset client * Regenerate automl client * Regenerate bigquerydatatransfer client * Regenerate bigquerystorage client * Regenerate bigtable client * Regenerate container client * Regenerate containeranalysis client * Regenerate dataproc client * Regenerate dialogflow client * Regenerate dlp client * Regenerate errorreporting client * Regenerate iamcredentials client * Regenerate iot client * Regenerate kms client * Regenerate language client * Regenerate logging client * Regenerate monitoring client * Regenerate os-login client * Regenerate pubsub client * Regenerate redis client * Regenerate securitycenter client * Regenerate speech client * Regenerate tasks client * Regenerate trace client * Regenerate video-intelligence client * Regenerate websecurityscanner client * Release google-cloud-java v0.78.0 (#4386) * Regenerate compute client (#4359) * Cloud Bigtable: tableAdmin sample (#4313) * tableAdmin sample and tests * comments added * files renamed and other edits * some changes in TableAdminExample and tests * fixed timestamp to base 16 * separate properties for bigtable.project and bigtable.instance * Regenerate scheduler client (#4294) * Regenerate spanner client (#4388) * Bump next snapshot (#4391) * Regenerate asset client (#4395) * Regenerate scheduler client (#4396) * Firestore: Include a trailing /documents on root resource paths (#4352) This is required for v1 and accepted in v1beta1. Port of googleapis/nodejs-firestore@52c7381 * Release v0.79.0 (#4402) * Removing some unused dependencies (#4385) * Removing some unused dependencies Also, reducing scope of auto-value to provided. * Restoring Firestore auto-value * Removing more instances of easymock. * Removing non-deprecated uses of joda time. (#4351) * Removing non-deprecated uses of joda time. This works towards #3482 * Update pom.xml * Ran `mvn com.coveo:fmt-maven-plugin:format` * Fixing a bad merge * Bump next snapshot (#4405) * fix shading (#4406) * Fixing some deprecation warnings (#4390) * fixing some deprecation warnings * updated comment * BigQuery : Fix Location configurable at BigQueryOptions (#4329) * Fix Location configurable from BigQueryOptions * modified code * modified code and add test case * removed unused location * Generate Firestore API v1 (#4410) * Bigtable: make row & cell ordering more consistent. (#4421) * Bigtable: make row & cell ordering more consistent. * RowCells should always be ordered in lexicographically by family, then qualifier and finally by reverse chronological order * Although rows will always be ordered lexicographically by row key, they should not implement Comparable to avoid confusion when compareTo() == 0 and equals() is false. Instead that ordering was moved to a separate comparator. * Add helpers to filter cells by family & qualifier * tweaks * code style * code style
sduskis
pushed a commit
that referenced
this pull request
Feb 8, 2019
* Bump next snapshot (#4300) * Cloud Bigtable: instanceAdmin sample (#4299) * instanceAdmin sample and tests * comments added * instanceAdminExample changes * renamed variables * changes * Regenerate securitycenter client (#4311) * 3540: Used transformer for shaded plugin (#4305) * Added Query#fromProto to convert protobuf ReadRowsRequest (#4291) * Adding Query.fromProto * Adding Query.fromProto * adding changes as per feedback * updated javadoc as per feedback * reformatted Query & QueryTest.java * Bigquery: corrected equality check on subclasses of StringEnumValue (#4283) * Bigquery: corrected equality check on subclasses of StringEnumValue * update format for FieldTest.java * Fix #4269 update metata url to FQDN (#4278) * 4152: Added checking PAGE_TOKEN from options (#4303) * 3918: Added checking of attributes size for 0. Added unit test (#4304) * Bigtable: Merge bigtable-admin into the bigtable artifact (#4307) * Bigtable: merge bigtable-admin into the bigtable artifact * split exclusion change for future PR * fix admin pathes * fix deprecation warning * fix synth script * fix generated file * temporarily re-add kokoro scripts (and point them to the merged artifact) until the jobs have been removed * Remove dep from examples * fix admin integration tests * revert stray fix (will be added to a separate PR) * fix int tests * don't double format the code * fix up docs * Regenerate PubSub: documentation updates (#4293) * Regenerate monitoring client (#4316) * Regenerate bigtable client (#4318) * 3822: Fixed setDisableGZipContent call (#4314) * 3822: Fixed setDisableGZipContent call * 3822: Changes after review * Regenerate speech client: enable multi-channel features (#4317) * Release v0.77.0 (#4324) * Created enum Region.java to retrieve Regions without doing request. (#4309) * Bump next snapshot (#4325) * Bigtable: fix handling of unset rpc timeouts (#4323) When the rpc timeout is zero, then it should be treated as disabled not actual 0 * Bigtable: Expose single row read settings (#4321) * exposing ReadRowSettings thru BigtableDataSettings * fixed typo error * Regenerate compute client (#4327) * Regenerate speech client (#4328) * Update README version to use bom version (#4322) * BigQuery: Fix NPE for null table schema fields (#4338) * Add failing test for empty table schema * Fix NPE if table schema returns null for fields * Bump gax, grpc & opencensus version (#4336) * Cloud Bigtable: HelloWorld sample updates (#4339) * comments added in HelloWorld and ITHelloWorld * removed typsafe name * separate properties for bigtable.project and bigtable.instance * separate properties for bigtable.project and bigtable.instance (#4346) * Pub/Sub: default values in batch settings comments (#4350) * Pub/Sub: default values in batch settings comments Ref: https://github.com/googleapis/gax-java/blob/master/gax/src/main/java/com/google/api/gax/batching/BatchingSettings.java * Verbose comment * Update maven-surefire-plugin to 3.0.0-M3 to fix Java 8 classloader (#4344) * Update maven-surefire-plugin to 3.0.0-M3 to fix Java 8 classloader * Update failsafe plugin to 3.0.0-M3 too * Specify maven-surefire-plugin version in bigtable-emulator * Bigtable: Fixing a typo for KeyOffSet#geyKey to getKey (#4342) * Regenerate spanner client (#4332) * Spanner: remove junit from compile dependencies (#4334) * Firestore: change timestampsInSnapshots default to true. (#4353) BREAKING CHANGE: The `areTimestampsInSnapshotsEnabled()` setting is now enabled by default so timestamp fields read from a `DocumentSnapshot` will be returned as `Timestamp` objects instead of `Date`. Any code expecting to receive a `Date` object must be updated. * Regenerate clients with updated copyright year (#4382) * Regenerate asset client * Regenerate automl client * Regenerate bigquerydatatransfer client * Regenerate bigquerystorage client * Regenerate bigtable client * Regenerate container client * Regenerate containeranalysis client * Regenerate dataproc client * Regenerate dialogflow client * Regenerate dlp client * Regenerate errorreporting client * Regenerate iamcredentials client * Regenerate iot client * Regenerate kms client * Regenerate language client * Regenerate logging client * Regenerate monitoring client * Regenerate os-login client * Regenerate pubsub client * Regenerate redis client * Regenerate securitycenter client * Regenerate speech client * Regenerate tasks client * Regenerate trace client * Regenerate video-intelligence client * Regenerate websecurityscanner client * Release google-cloud-java v0.78.0 (#4386) * Regenerate compute client (#4359) * Cloud Bigtable: tableAdmin sample (#4313) * tableAdmin sample and tests * comments added * files renamed and other edits * some changes in TableAdminExample and tests * fixed timestamp to base 16 * separate properties for bigtable.project and bigtable.instance * Regenerate scheduler client (#4294) * Regenerate spanner client (#4388) * Bump next snapshot (#4391) * Regenerate asset client (#4395) * Regenerate scheduler client (#4396) * Firestore: Include a trailing /documents on root resource paths (#4352) This is required for v1 and accepted in v1beta1. Port of googleapis/nodejs-firestore@52c7381 * Release v0.79.0 (#4402) * Removing some unused dependencies (#4385) * Removing some unused dependencies Also, reducing scope of auto-value to provided. * Restoring Firestore auto-value * Removing more instances of easymock. * Removing non-deprecated uses of joda time. (#4351) * Removing non-deprecated uses of joda time. This works towards #3482 * Update pom.xml * Ran `mvn com.coveo:fmt-maven-plugin:format` * Fixing a bad merge * Bump next snapshot (#4405) * fix shading (#4406) * Fixing some deprecation warnings (#4390) * fixing some deprecation warnings * updated comment * BigQuery : Fix Location configurable at BigQueryOptions (#4329) * Fix Location configurable from BigQueryOptions * modified code * modified code and add test case * removed unused location * Generate Firestore API v1 (#4410) * Bigtable: make row & cell ordering more consistent. (#4421) * Bigtable: make row & cell ordering more consistent. * RowCells should always be ordered in lexicographically by family, then qualifier and finally by reverse chronological order * Although rows will always be ordered lexicographically by row key, they should not implement Comparable to avoid confusion when compareTo() == 0 and equals() is false. Instead that ordering was moved to a separate comparator. * Add helpers to filter cells by family & qualifier * tweaks * code style * code style * Fix code formatting (#4437) * Regenerate compute client (#4399) * Regenerate compute client (#4444) * Firestore: Migrate client to use v1 gapic client / protos. (#4419) * Regenerate asset client (#4426) * Regenerate bigtable client (#4427) * Regenerate dataproc client (#4428) * Regenerate speech client (#4422) * Regenerate dialogflow client (#4429) * Regenerate redis client (#4431) * Regenerate securitycenter client (#4432) * Regenerate tasks client (#4411) * Fix usages of any(<Primitive>.class) matchers (#4453) In Mockito 2, if a method expects a primitive type, but an any(<Primitive>.class) matcher is used in its place, it will throw an error. To prepare for this upcoming breakage, change all existing any(<Primitive>.class) matchers to use the correct any<Primitive>() matcher. * Regenerate video-intelligence client (#4434) * Regenerate automl client (#4418) * Regenerate automl client (#4455) * Regenerate speech client (#4456) * add comments (#4441) * Update some default retry setting values (#4425) Ref: https://github.com/googleapis/google-cloud-java/blob/ed7bc857a05b34eed6be182d4798a62bf09cd394/google-cloud-clients/google-cloud-pubsub/src/main/java/com/google/cloud/pubsub/v1/Publisher.java#L497 * Implement BucketPolicyOnly (#4404) * implement BucketPolicyOnly * [Storage] Bucket Policy Only samples (#4408) * Humble beginnings for BPO samples with tests * Update samples per client changes * Fix format issue * Lint fix * Bigtable: make request context serializable (#4459) * Bigtable: expose helper to convert proto rows into model rows (#4458) This is needed for hbase adapter transition * Regenerate texttospeech client (#4333) * Regenerate compute client (#4462) * Skip integration tests if no changes (#4392) * Add environment variable for allowing skipping ITs * Skip integration tests if there are no changes in that directory * Also check google-cloud-core client and the grpc-api generated models * update TESTING.md (#4409) * Regenerate spanner client (#4433) * Regenerate texttospeech client (#4463) * 3534: Made HttpTransportOptions constructor public. (#4439) * use gax 1.38.0 (#4460) * Regenerate firestore client (#4348) * Empty table results can have a schema (#4185) * Update EmptyTableResult.java * Update Job.java * add test case * 4273: Added a new create method to pass offset and length of sub array. (#4407) * 4273: Added a new create method to pass offset and lenght of sub array. * 4273: Fixed codeformat error. * 4273: Rephrased a comment. * 4273: Added a new integration test using the new createBlobWithSubArrayFromByteArray code snippet. * Add firestore grpc and proto versions to versions.txt (#4464) * Release v0.80.0 (#4465) * Bump next snapshot (#4467) * Adding Cloud Bigtable Mutation fromProto (#4461) * Adding Cloud Bigtable Mutation fromProto * Fixing formatting * Addressing comments `fromProto` becomes `fromProtoUnsafe` * Fixing lint issues. * adding a warning to `fromProtoUnsafe` * Regenerate websecurityscanner client (#4435)
kamalaboulhosn
added a commit
that referenced
this pull request
Jul 9, 2019
* Merge from master. (#4468) * Bump next snapshot (#4300) * Cloud Bigtable: instanceAdmin sample (#4299) * instanceAdmin sample and tests * comments added * instanceAdminExample changes * renamed variables * changes * Regenerate securitycenter client (#4311) * 3540: Used transformer for shaded plugin (#4305) * Added Query#fromProto to convert protobuf ReadRowsRequest (#4291) * Adding Query.fromProto * Adding Query.fromProto * adding changes as per feedback * updated javadoc as per feedback * reformatted Query & QueryTest.java * Bigquery: corrected equality check on subclasses of StringEnumValue (#4283) * Bigquery: corrected equality check on subclasses of StringEnumValue * update format for FieldTest.java * Fix #4269 update metata url to FQDN (#4278) * 4152: Added checking PAGE_TOKEN from options (#4303) * 3918: Added checking of attributes size for 0. Added unit test (#4304) * Bigtable: Merge bigtable-admin into the bigtable artifact (#4307) * Bigtable: merge bigtable-admin into the bigtable artifact * split exclusion change for future PR * fix admin pathes * fix deprecation warning * fix synth script * fix generated file * temporarily re-add kokoro scripts (and point them to the merged artifact) until the jobs have been removed * Remove dep from examples * fix admin integration tests * revert stray fix (will be added to a separate PR) * fix int tests * don't double format the code * fix up docs * Regenerate PubSub: documentation updates (#4293) * Regenerate monitoring client (#4316) * Regenerate bigtable client (#4318) * 3822: Fixed setDisableGZipContent call (#4314) * 3822: Fixed setDisableGZipContent call * 3822: Changes after review * Regenerate speech client: enable multi-channel features (#4317) * Release v0.77.0 (#4324) * Created enum Region.java to retrieve Regions without doing request. (#4309) * Bump next snapshot (#4325) * Bigtable: fix handling of unset rpc timeouts (#4323) When the rpc timeout is zero, then it should be treated as disabled not actual 0 * Bigtable: Expose single row read settings (#4321) * exposing ReadRowSettings thru BigtableDataSettings * fixed typo error * Regenerate compute client (#4327) * Regenerate speech client (#4328) * Update README version to use bom version (#4322) * BigQuery: Fix NPE for null table schema fields (#4338) * Add failing test for empty table schema * Fix NPE if table schema returns null for fields * Bump gax, grpc & opencensus version (#4336) * Cloud Bigtable: HelloWorld sample updates (#4339) * comments added in HelloWorld and ITHelloWorld * removed typsafe name * separate properties for bigtable.project and bigtable.instance * separate properties for bigtable.project and bigtable.instance (#4346) * Pub/Sub: default values in batch settings comments (#4350) * Pub/Sub: default values in batch settings comments Ref: https://github.com/googleapis/gax-java/blob/master/gax/src/main/java/com/google/api/gax/batching/BatchingSettings.java * Verbose comment * Update maven-surefire-plugin to 3.0.0-M3 to fix Java 8 classloader (#4344) * Update maven-surefire-plugin to 3.0.0-M3 to fix Java 8 classloader * Update failsafe plugin to 3.0.0-M3 too * Specify maven-surefire-plugin version in bigtable-emulator * Bigtable: Fixing a typo for KeyOffSet#geyKey to getKey (#4342) * Regenerate spanner client (#4332) * Spanner: remove junit from compile dependencies (#4334) * Firestore: change timestampsInSnapshots default to true. (#4353) BREAKING CHANGE: The `areTimestampsInSnapshotsEnabled()` setting is now enabled by default so timestamp fields read from a `DocumentSnapshot` will be returned as `Timestamp` objects instead of `Date`. Any code expecting to receive a `Date` object must be updated. * Regenerate clients with updated copyright year (#4382) * Regenerate asset client * Regenerate automl client * Regenerate bigquerydatatransfer client * Regenerate bigquerystorage client * Regenerate bigtable client * Regenerate container client * Regenerate containeranalysis client * Regenerate dataproc client * Regenerate dialogflow client * Regenerate dlp client * Regenerate errorreporting client * Regenerate iamcredentials client * Regenerate iot client * Regenerate kms client * Regenerate language client * Regenerate logging client * Regenerate monitoring client * Regenerate os-login client * Regenerate pubsub client * Regenerate redis client * Regenerate securitycenter client * Regenerate speech client * Regenerate tasks client * Regenerate trace client * Regenerate video-intelligence client * Regenerate websecurityscanner client * Release google-cloud-java v0.78.0 (#4386) * Regenerate compute client (#4359) * Cloud Bigtable: tableAdmin sample (#4313) * tableAdmin sample and tests * comments added * files renamed and other edits * some changes in TableAdminExample and tests * fixed timestamp to base 16 * separate properties for bigtable.project and bigtable.instance * Regenerate scheduler client (#4294) * Regenerate spanner client (#4388) * Bump next snapshot (#4391) * Regenerate asset client (#4395) * Regenerate scheduler client (#4396) * Firestore: Include a trailing /documents on root resource paths (#4352) This is required for v1 and accepted in v1beta1. Port of googleapis/nodejs-firestore@52c7381 * Release v0.79.0 (#4402) * Removing some unused dependencies (#4385) * Removing some unused dependencies Also, reducing scope of auto-value to provided. * Restoring Firestore auto-value * Removing more instances of easymock. * Removing non-deprecated uses of joda time. (#4351) * Removing non-deprecated uses of joda time. This works towards #3482 * Update pom.xml * Ran `mvn com.coveo:fmt-maven-plugin:format` * Fixing a bad merge * Bump next snapshot (#4405) * fix shading (#4406) * Fixing some deprecation warnings (#4390) * fixing some deprecation warnings * updated comment * BigQuery : Fix Location configurable at BigQueryOptions (#4329) * Fix Location configurable from BigQueryOptions * modified code * modified code and add test case * removed unused location * Generate Firestore API v1 (#4410) * Bigtable: make row & cell ordering more consistent. (#4421) * Bigtable: make row & cell ordering more consistent. * RowCells should always be ordered in lexicographically by family, then qualifier and finally by reverse chronological order * Although rows will always be ordered lexicographically by row key, they should not implement Comparable to avoid confusion when compareTo() == 0 and equals() is false. Instead that ordering was moved to a separate comparator. * Add helpers to filter cells by family & qualifier * tweaks * code style * code style * Fix code formatting (#4437) * Regenerate compute client (#4399) * Regenerate compute client (#4444) * Firestore: Migrate client to use v1 gapic client / protos. (#4419) * Regenerate asset client (#4426) * Regenerate bigtable client (#4427) * Regenerate dataproc client (#4428) * Regenerate speech client (#4422) * Regenerate dialogflow client (#4429) * Regenerate redis client (#4431) * Regenerate securitycenter client (#4432) * Regenerate tasks client (#4411) * Fix usages of any(<Primitive>.class) matchers (#4453) In Mockito 2, if a method expects a primitive type, but an any(<Primitive>.class) matcher is used in its place, it will throw an error. To prepare for this upcoming breakage, change all existing any(<Primitive>.class) matchers to use the correct any<Primitive>() matcher. * Regenerate video-intelligence client (#4434) * Regenerate automl client (#4418) * Regenerate automl client (#4455) * Regenerate speech client (#4456) * add comments (#4441) * Update some default retry setting values (#4425) Ref: https://github.com/googleapis/google-cloud-java/blob/ed7bc857a05b34eed6be182d4798a62bf09cd394/google-cloud-clients/google-cloud-pubsub/src/main/java/com/google/cloud/pubsub/v1/Publisher.java#L497 * Implement BucketPolicyOnly (#4404) * implement BucketPolicyOnly * [Storage] Bucket Policy Only samples (#4408) * Humble beginnings for BPO samples with tests * Update samples per client changes * Fix format issue * Lint fix * Bigtable: make request context serializable (#4459) * Bigtable: expose helper to convert proto rows into model rows (#4458) This is needed for hbase adapter transition * Regenerate texttospeech client (#4333) * Regenerate compute client (#4462) * Skip integration tests if no changes (#4392) * Add environment variable for allowing skipping ITs * Skip integration tests if there are no changes in that directory * Also check google-cloud-core client and the grpc-api generated models * update TESTING.md (#4409) * Regenerate spanner client (#4433) * Regenerate texttospeech client (#4463) * 3534: Made HttpTransportOptions constructor public. (#4439) * use gax 1.38.0 (#4460) * Regenerate firestore client (#4348) * Empty table results can have a schema (#4185) * Update EmptyTableResult.java * Update Job.java * add test case * 4273: Added a new create method to pass offset and length of sub array. (#4407) * 4273: Added a new create method to pass offset and lenght of sub array. * 4273: Fixed codeformat error. * 4273: Rephrased a comment. * 4273: Added a new integration test using the new createBlobWithSubArrayFromByteArray code snippet. * Add firestore grpc and proto versions to versions.txt (#4464) * Release v0.80.0 (#4465) * Bump next snapshot (#4467) * Adding Cloud Bigtable Mutation fromProto (#4461) * Adding Cloud Bigtable Mutation fromProto * Fixing formatting * Addressing comments `fromProto` becomes `fromProtoUnsafe` * Fixing lint issues. * adding a warning to `fromProtoUnsafe` * Regenerate websecurityscanner client (#4435) * Add SequentialExecutorService that assists Pub/Sub to publish messages sequentially (#4315) * Add SequentialExecutorService * Add headers, make methods package private, and other review changes. * Fix formatting of SequentialExecutorService classes. * Add ordered publishing support to Cloud Pub/Sub (#4474) * Add ordering key fields to proto. We are locally committing the proto with ordering keys for now. Once we are prepared to release this feature, we can use the default, released proto. * Add support for ordered publishing * Add comments about ordering key properties. * Change retry codes and add checks for a null ordering key. * Pubsub ordering keys subscriber (#4515) * Add ordering key fields to proto. We are locally committing the proto with ordering keys for now. Once we are prepared to release this feature, we can use the default, released proto. * Add support for ordered publishing * Add comments about ordering key properties. * Change retry codes and add checks for a null ordering key. * Add support for ordered delivery to subscribers * Change the order of publishing batches when a large message is requested; Add unit tests (#4578) * Merge from master; Resolve conflicts (#4943) * Scale the system executor provider with the number of pull channels opened. (#4592) Make the SubscriberStubSettings refer to the user provided executor provider instead of a fixed instantiation of it. If the user provides an InstantiatingExecutorProvider instead of a FixedExecutorProvider, this will actually instantiate more than one as the user would expect. It will still only instantiate one for all connections to share, and will do so until the next PR which will make them have different stub instantiations. * Using the google-cloud-http-bom (#4594) * Using the latest version of truth (#4597) * Using a much newer version of truth * Upgrading annotations in google-api-grpc * Regenerate websecurityscanner client (#4614) * Regenerate vision client (#4613) * Regenerate video-intelligence client (#4612) * Regenerate tasks client (#4611) * Regenerate securitycenter client (#4610) * BigQuery: Fix useAvroLogicalTypes option in LoadJobConfiguration And WriteChannelConfiguration. (#4615) * Fix useAvroLogicalTypes in LoadJobConfiguration * Fix comment and added useAvroLogicalTypes in WriteChannelConfiguration * Fix comment * Fix code format * add propagateTimeout in SpannerExceptionFactory (#4598) - Add a helper method propagateTimeout in SpannerExceptionFactory to easily transform a TimeoutException to a SpannerException. - Propagate ApiExceptions a bit more nicely. * Regenerate scheduler client (#4609) * Regenerate pubsub client (#4608) * Regenerate os-login client (#4607) * Regenerate dlp client (#4603) * Regenerate iamcredentials client (#4605) * Regenerate bigquerystorage client (#4601) * Regenerate dataproc client (#4602) * Regenerate bigquerydatatransfer client (#4600) * Regenerate language client (#4606) * Regenerate firestore client (#4604) * Set the isDirectory flag appropriately (#4616) Set the isDirectory flag appropriately when using the currentDirectory() option * Extract the single message processing functionality from processOutstandingBatches. (#4618) * Upgrading grpc in google-api-grpc (#4593) * Prepare for KMS GA release - upgrade versions to 1.0.0. (#4581) * Adding vision v1p4beta1 (#4584) * Adding vision v1p4beta1 * Updating more pom.xmls * Fixing formatting issues * Clean up MessageDispatcher by changing processOutstandingBatches to explicitly loop instead of while(true) with breaks. There is now only 1 explicit return and 1 runtime error. (#4619) * Release google-cloud-java v0.82.0 (#4621) * Release v0.82.0 * Change KMS versions to 1.0.0. * Remove global synchronization from MessageDispatcher. (#4620) * Remove global synchronization from MessageDispatcher. Now that this uses a LinkedBlockingDeque for batches, this is no longer necessary. * Run code format. * Bump next snapshot (#4623) * Regenerate vision client (#4625) * Regenerate dataproc client (#4634) * Regenerate dialogflow client (#4635) * Regenerate firestore client (#4638) * Regenerate iot client (#4639) * Regenerate monitoring client (#4642) * Regenerate bigtable client (#4631) * Regenerate errorreporting client (#4637) * Regenerate pubsub client (#4643) * Regenerate automl client (#4629) * Regenerate redis client (#4644) * Regenerate bigquerydatatransfer client (#4630) * Regenerate scheduler client (#4645) * Regenerate kms client (#4640) * Regenerate vision client (#4650) * Regenerate websecurityscanner client (#4651) * Regenerate containeranalysis client (#4633) * Regenerate dlp client (#4636) * Regenerate trace client (#4649) * Regenerate spanner client (#4647) * Regenerate logging client (#4641) * Regenerate tasks client (#4648) * Regenerate securitycenter client (#4646) * Bump gax to 1.42.0 (#4624) Also regenerate the Compute client to match the newest version of gax. * Batch dml mainline (#4653) * Cloud Spanner Batch DML implementation and integration tests. (#45) * Fix the file header of the newly added classes. (#46) * Fix RPC interface mismatch after GAPIC migration. * Address review comment. * Fix code format with mvn com.coveo:fmt-maven-plugin:format. * Update year in file headers. * BigQuery: add long term storage bytes to standard table definition. (#4387) * BigQuery: Add long term storage bytes for managed tables. * formatting * let maven format the things * plumb this upwards into Table/TableInfo * return * assertion mismatch * Update TableInfoTest.java * Regenerate compute client (#4662) * Add Cloud Security Center v1 API. (#4659) * Add Cloud Security Center v1 API. * Add securitycenter to google-cloud-bom pom.xml * Fixing code format * Adding FieldValue.increment() (#4018) * DO NOT MERGE: Adding FieldValue.increment() * Use "increment" Proto naming * Spanner: Throw exception when SSLHandshakeException occurs instead of infinite retry loop (#4663) * #3889 throw exception when an SSLHandshakeException occurs SSLHandshakeExceptions are not retryable, as it is most probably an indication that the client does not accept the server certificate. * #3889 added test for retryability of SSLHandshakeException * fixed formatting * Add Cloud Scheduler v1 API. (#4658) * Add Cloud Scheduler v1 API. * Fixes to google-cloud-bom pom.xml * Add proto to scheduler pom.xml * Fix Timestamp.parseTimestamp. (#4656) * Fix parseTimestamp docs * Fix timestamp without timezone offset * Fix test cases related to timestamp.parseTimestamp * added test case * Fix timestampParser and added ZoneOffset in timestampParser * Release google-cloud-java v0.83.0 (#4665) * Regenerate securitycenter client (#4667) * Bump next snapshot (#4666) * Change each StreamingSubscriberConnection to have its own executor by default. (#4622) * Change each StreamingSubscriberConnection to have its own executor by default. This increases throughput by reducing contention on the executor queue mutex and makes the Subscriber implementation more accurately reflect the users intent when an InstantiatingExecutorProvider is passed. * Add a comment for executorProvider and alarmsExecutor. * Bigtable: Remove reference to deprecated typesafe name (#4671) * Add Cloud Video Intelligence v1p3beta1 API. (#4669) * Add Cloud Video Intelligence v1p3beta1 API. * Fix code formatting. * Regenerate language client (#4676) * Regenerate securitycenter client (#4677) * Regenerate firestore client (#4686) * #4685 Spanner now extends AutoCloseable (#4687) * Update speech readme to point to v1 javadoc. (#4693) * Fix pendingWrite race condition (#4696) pendingWrites.add() was not guaranteed to be called before pendingWrites.remove() * Optimize pendingWrites (#4697) Reduce contention on pendingWrites by using a ConcurrentHashMap instead. * Better explain how to use explicit credentials (#4694) * Better explain how to use explicit credentials This pull request updates the documentation and adds an example. * Run auto-formatter mvn com.coveo:fmt-maven-plugin:format * Updating Copyright year on UseExplicitCredentials * Add Cloud Talent Solution API. (#4699) * Add Talent API * Add Talent API * Add Talent API * Add talent API * reformat * Update pom.xml * Update pom.xml * Add MDC support in Logback appender (#4477) * Firestore: Update CustomClassMapper (#4675) * Firestore: Update CustomClassMapper * Adding Unit tests * Lint fix * Regenerate securitycenter client (#4704) * Regenerate talent client (#4705) * BigQuery: Added missing partitioned fields to listTables. (#4701) * Add missing partitioned fields to listTables * Fix table delete in finally block * Fix test failing * OpenCensus Support for Cloud Pub/Sub (#4240) * Adds OpenCensus context propagation to Publisher and Subscriber. * Updates OpenCensus attribute keys so that they will be propagated by CPS. * Addresses reviewer comments by fixing build files and using only defined annotations. * Updates build dependencies and copyright date. * Fixes typo. * Removes encoding of OpenCensus tags. Will re-enable once text encoding spec has been finalized (census-instrumentation/opencensus-specs#65). * Updates encoding of SpanContext to use W3C specified encoding; Also preserves sampling decision from the publisher in the subscriber. * Adds unit test for OpenCensusUtil. * Adds unit test for OpenCensusUtil. * Updates OpenCensus integration to use a generic MessageTransform. * Removes now-unused private constant. * Update pom.xml * Marking setTransform as BetaApi * Fixes for formatting issues. * Release v0.84.0 (#4713) * Bump next snapshot (#4715) * Fix comment in grpc-google-cloud-firestore-v1 POM (#4717) Fixing a copy&paste typo * Regenerate firestore client (#4719) * Upgrade common protos dependency to 1.15.0. (#4722) * V1 Admin API for Firestore (#4716) * Manually regenerate Redis (#4723) * Manually regenerate Redis * Fixing a formatting issue * 1.x is stable (#4724) @sduskis * Regenerate bigquerystorage client (#4730) * Regenerate pubsub client (#4732) * Added feature to accept Firestore Value. (#4709) * Fix firestore value * Added IT test * try to fix flaky test (#4733) (cherry picked from commit 8255a9b) * Regenerate compute client (#4731) * Replacing GoogleCloudPlatform with googleapis in docs. (#4707) * Replacing GoogleCloudPlatform with googleapis in docs. * Fixing formatting issues. * Regenerate Redis client (#4738) * redis changes with googleapis changes for STATIC_TYPES * foo * revert noop * fix default value of maxSessions in document of setMaxSessions (#4728) * Spanner: Added extra documentation to the DatabaseClient.singleUse methods (#4721) * added extra documentation to the singleUse methods #4212 * changed formatting manually and re-run mvn formatter * Regenerate Firestore clients (#4741) * add diff * linter * Regenerate iamcredentials client (#4746) * Regenerate monitoring client (#4747) * Refresh Monitoring client (#4744) * regen monitoring with STATIC_TYPES * regen again but with java_package * regen again with enable_string_formatting_functions * regen again * the OuterClass is generated because the proto file contains the amessage of the same name * no changes in protos or gapics, just refresh * refresh but with proto changes only * Add Snippets for working with Assets in Cloud Security Command Center. (#4690) * Add Snippets for working with Assets in Cloud Security Command Center. - I missed instruction for how to add a new key, but I believe a new account is needed, so prestaged assets can be queried. * Address code review comments. * remove securitycenter-it.cfg * update comments * fix string remove firewall reference * Fix format * fix format * Address comments and fix bad docs/alignment with python example * Fix print description * Updates per rubrics * fix warnings * Add Apache Headers * remove unused comment * Manually regenerating the talent v4beta1 API (#4750) * Manually regenerating the talent v4beta1 API ResumeService was removed from the protos, requiring manual deletion of all related classes. * Fixing formatting. * Pass through header provider in BQ storage client settings. (#4740) * Pass through header provider in BQ storage client settings. This change modifies the enhanced stub for the BigQuery storage client to pass through the user-specified header provider to the storage settings base class at creationt ime. This addresses an issue where user-specified headers were being ignored when creating a client object. * Apply formatting plugin * Set up the in-proc server only once * It's 2019 now * changes for iamcreds with STATIC_TYPES (#4745) * Regenerate asset client (#4758) * Regenerate automl client (#4759) * Regenerate dlp client (#4765) * Regenerate bigtable client (#4760) * Regenerate logging client (#4769) * Regenerate monitoring client (#4770) * Regenerate pubsub client (#4772) * Regenerate container client (#4761) * Regenerate os-login client (#4771) * Regenerate errorreporting client (#4766) * Regenerate containeranalysis client (#4762) * Regenerate talent client (#4777) * Regenerate securitycenter client (#4775) * Regenerate redis client (#4773) * Regenerate scheduler client (#4774) * Regenerate iamcredentials client (#4768) * Regenerate dialogflow client (#4764) * Regenerate firestore client (#4767) * Regenerate spanner client (#4776) * Regenerate trace client (#4778) * Regenerate dataproc client (#4763) * Regenerate vision client (#4779) * Regenerate websecurityscanner client (#4780) * regen (#4755) * Upgrade google.auth.version to 0.15.0. (#4743) * Upgrade google.auth.version to 0.14.0. * Upgrade google.auth.version to 0.14.0. * Upgrade google.auth.version to 0.15.0. * allow specifying a custom credentials file for integration tests (#4782) * Regenerate containeranalysis client (#4792) * Regenerate dataproc client (#4793) * Regenerate datastore client (#4794) * Regenerate asset client (#4786) * Regenerate iamcredentials client (#4799) * Regenerate bigquerydatatransfer client (#4788) * Regenerate language client (#4802) * Regenerate kms client (#4801) * Regenerate bigtable client (#4790) * Regenerate monitoring client (#4804) * Regenerate bigquerystorage client (#4789) * Regenerate speech client (#4811) * Regenerate securitycenter client (#4809) * Regenerate spanner client (#4810) * Regenerate redis client (#4807) * Regenerate container client (#4791) * Regenerate scheduler client (#4808) * Regenerate os-login client (#4805) * Regenerate errorreporting client (#4797) * Regenerate websecurityscanner client (#4818) * Regenerate logging client (#4803) * Regenerate texttospeech client (#4814) * Regenerate tasks client (#4813) * Regenerate dialogflow client (#4795) * Regenerate trace client (#4815) * Upgrade protobuf version to 3.7.0. (#4819) * Upgrade protobuf version to 3.7.0. * Upgrade protobuf version to 3.7.1. * Upgrade protobuf version to 3.7.0. * Release google-cloud-java v0.85.0 (#4820) * Bump snapshot (#4821) * Regenerate automl client (#4822) * Regenerate dialogflow client (#4823) * Regenerate dlp client (#4824) * Regenerate firestore client (#4825) * Regenerate iot client (#4826) * Regenerate pubsub client (#4827) * Regenerate talent client (#4828) * Regenerate video-intelligence client (#4829) * Regenerate vision client (#4830) * Add Cloud Asset v1 API. (#4834) * Upgrade gax to 1.43.0. (#4836) * Upgrade gax to 1.43.0. * code format * Regenerate iamcredentials client (#4849) * Regenerate iot client (#4850) * Regenerate kms client (#4851) * Regenerate texttospeech client (#4864) * Regenerate tasks client (#4863) * Regenerate talent client (#4862) * Regenerate bigquerystorage client (#4840) * Regenerate bigquerydatatransfer client (#4839) * Regenerate automl client (#4838) * Regenerate language client (#4852) * Regenerate monitoring client (#4854) * Regenerate pubsub client (#4856) * Regenerate redis client (#4857) * Regenerate scheduler client (#4858) * Regenerate securitycenter client (#4859) * Regenerate spanner client (#4860) * Regenerate trace client (#4865) * Regenerate video-intelligence client (#4866) * Regenerate websecurityscanner client (#4868) * Regenerate vision client (#4867) * Regenerate speech client (#4861) * Regenerate os-login client (#4855) * Regenerate logging client (#4853) * Regenerate firestore client (#4848) * Regenerate containeranalysis client (#4843) * Regenerate container client (#4842) * Regenerate bigtable client (#4841) * Regenerate dataproc client (#4844) * Regenerate dlp client (#4846) * Regenerate errorreporting client (#4847) * google-cloud-logging-logback: Allow user to specify custom LoggingOptions (#4729) google-cloud-logging-logback: #3215 allow user to specify custom LoggingOptions * Regenerate asset client (#4837) * Regenerate dialogflow client (#4845) * Add translate v3beta1 API. (#4870) * translate v3beta1 * readme * Adding datalabeling. (#4872) * Adding datalabeling. * Fixing formatting. * Adding datalabeling readme. * Fixing Readme. * Add tasks v2 client (#4879) * auto close input stream (#4878) * Add Cloud Web Risk client. (#4881) * Add Cloud Web Risk client. * code format fix * Add note on whitelist / signing up for Beta. * Delete stray preposition * Regenerate datalabeling client (#4883) * Regenerate automl client (#4882) * Regenerate talent client (#4884) * fix flaky TransactionManager tests (#4897) Transactions can always be aborted by Cloud Spanner, and all transaction code should therefore be surrounded by with abort safeguarding. * Change .yaml file name for webrisk client. (#4898) * Adding the translate beta samples (#4880) * Adding the translate beta samples * Fixing lint issues * Fixing coding format/style * Fixing coding format/style * Release v0.86.0 (#4899) * Bump next snapshot (#4900) * Regenerate bigquerystorage client (#4902) * Regenerate language client (#4903) * Regenerate webrisk client (#4904) * Storage: Add V4 signing support (#4692) * Add support for V4 signing * (storage) WIP: Add V4 signing support * (storage) Add V4 signing support * Add V4 samples (#4753) * storage: fix v4 samples (#4754) * Add V4 samples * Match C++ samples * Add missing import * Add web risk to list of clients. (#4901) * Release v0.87.0 (#4907) * Bump next snapshot (#4908) * Regenerate tasks client (#4912) * Regenerate scheduler client (#4911) * Regenerate redis client (#4910) * Regenerate compute client (#4909) * Spanner: Refactor SpannerImpl - Move AbstractResultSet to separate file (#4891) Refactor SpannerImpl: Move AbstractResultSet to separate file * Spanner: Refactor SpannerImpl - Move AbstractReadContext to separate file (#4890) Spanner: Refactor SpannerImpl - Move AbstractReadContext to separate file * Regenerate language client (#4921) * Spanner: Refactor SpannerImpl - Move DatabaseAdminClientImpl to separate file (#4892) * refactor SpannerImpl: move DatabaseAdminClientImpl to separate file * removed unnecessary imports * Bigquery : Fix close write channel (#4924) * close write channel * Update BigQuerySnippets.java * Update BigQuerySnippets.java * Update BigQuerySnippets.java * Spanner: Refactor SpannerImpl - Move InstanceAdminClient to separate file (#4893) refactor SpannerImpl: move InstanceAdminClient to separate file * refactor SpannerImpl: move PartitionedDmlTransaction to separate file (#4894) * BigQuery: Added listPartitions. (#4923) * added listPartitions * added unit test * modified code * modified unit test for listPartitions * added integration test * Fix table name * Fix integration test * Regenerate dialogflow client (#4928) * Regenerate securitycenter client (#4929) * Regenerate trace client (#4931) * google-cloud-core: Optimize Date.parseDate(String) (#4920) * optimized date parsing Date parsing using a regular expression is slower than a specific implementation for the only format that is supported. * removed IntParser and added test cases * check for invalid date/year/month/day * Spanner: Refactor SpannerImpl - Move TransactionRunnerImpl to separate file (#4896) refactor SpannerImpl: move TransactionRunnerImpl to separate file * Regenerate securitycenter client (#4937) * storage: Un-Ignore testRotateFromCustomerEncryptionToKmsKeyWithCustomerEncryption (#4936) * Adding support to custom ports on LocalDatastoreHelper (#4933) (#4935) * Storage : Fix manage resumeable signedURL uploads. (#4874) * commit for manage resumeable signedURL uploads #2462 * for manage resumeable signedURL uploads #2462 * fix comment * fix ITStorageTest case written for upload using signURL * fix format * fix BlobWriteChannel constructor changes. * fix signURL validation. * fix format * signurl rename to signedURL , firstnonnull check removed,signedURL validation with googleacessid and expires field also. * signedURL validation with googleacessid and expires field also. * fix forsignedURL validation with V4 Signing support. * fix forproviding example of writing content using signedURL through Writer. * fix forStorageRpc open method argument change. * fix forStorageRpc open method doc comment changes. * cloud-storage-contrib-nio: Add support for newFileChannel (#4718) add support for newFileChannel(...) * Change MessageDispatcher to be synchronous instead of asynchronous. (#4916) * Change MessageDispatcher to be synchronous instead of asynchronous. This removes the failure mode described in #2452 that can occur when MaxOutstandingElementCount is low and there is more than one connection. In this case, it is possible for an individual MessageDispatcher to have no outstanding in-flight messages, but also be blocked by flow control with a whole new batch outstanding. In this case, it will never make progress on that batch since it will never receive another batch and the queue was made to not be shared in #4590, so the batch will never be pulled off by another MessageDispatcher. By changing this to use a blocking flow controller, this will never happen, as each batch will synchronously wait until it is allowed by flow control before being processed. * Run mvn com.coveo:fmt-maven-plugin:format * Remove unused code * Update Publisher.java * Fixing format * Refactoring of the Pub/Sub Ordering keys client (#4962) * [WIP] Refactoring SequentialExecutorService (#4969) * Refactoring SequentialExecutorService Step 1: - create a `CallbackExecutor` and `AutoExecutor` as subclasses of SequentialExecutor. * Adding CallbackExecutor.submit() for encapsulation * Moving resume into `CallbackExecutor` * create an abstract class for execute(key, deque) This removest the last bit of code that directly used the TaskCompleteAction enum. * Running the formatter. * Merged with the formatting changes. (#4978) * Fixng a bad merge. * Refactoring SequentialExecutorService.java (#4979) - Marking methods as private - renaming variables from `finalTasks` to `tasks` for clarity - reducing code duplication by calling `execute` * Cleaning up generics in SequentialExecutorService (#4982) * Adding comments to CallbackExecutor.submit (#4981) * Adding comments to CallbackExecutor.submit * Fixing the merge * Exposing AutoExecutor and CallbackExecutor directly (#4983) * More refactoring to SequentialExecutor (#4984) - Moving common async code into a new method: `callNextTaskAsync()` - Adding a `postTaskExecution()` method which is useful for `AutoExecutor` * SequentialExecutorService.callNextTaskAsync only uses key (#4992) 1. Removing the `Deque` parameter from `SequentialExecutorService.callNextTaskAsync()` and looking it up the `Deque` in the method 2. Simplify `AutoExecutor` and `CallbackExecutor` after the refactoring in 1). * SequentialExecutorService now uses generics for Runnables. * Using a Queue instead of Deque. Queues are FIFO, Deque don't guaratee which end the value is removed from. * Renaming a variable in Publisher * More refactoring to SequentialExecutorService. * Running the formatter * Reformatting the publisher * Pub/Sub: publishAll defers sending batches. * Reverting last change to publishAllOutstanding * Cleaning up messageBatch entries * Alarms now wait to publish until previous bundle completes * Using Preconditions in Publisher. * The Publisher's callback should happen second The `SequentialExecutor`'s future callback needs to happen before the `Publisher`'s callback, in case the `SequentialExecutor` cancels. * Fixing a flaky test The `testLargeMessagesDoNotReorderBatches()` test had a flaky test, depending on how unrelated threads complete. Removing the check for the behavior of the unrelated thread. * Adding resume publish. (#5046) * Adding resume publish. * Addressing comments * Fixing formatting issues * Adding comments * Fixing formatting * Ordering keys setter is now package private This will allow us to merge this branch into master. * Cleanup before merge to main line - Removing EnumSet from Status codes - Adding `@BetaApi` to `resumePublish`, and making it package private. * Fixing lint * publish() doesn't use an executor. * Moving the publish RPC call into the lock Also,removing a comment that no longer applies. * Updating GAX in clients to 1.45.0 for docs. * Fix publisher throughput when there are no ordering keys. (#5607) * Fix publisher throughput when there are no ordering keys. * Remove batch from iterator if we are going to process it. * Make appropriate things private, remove unnecessary call to publishAllWithoutInflight, and fix unit tests accordingly. * Fix publisher formatting * Experimental removal of test to see if it is the one causing timeouts. * Fix comment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
api: bigtable
Issues related to the Bigtable API.
cla: yes
This human has signed the Contributor License Agreement.
type: bug
Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When the rpc timeout is zero, then it should be treated as disabled not actual 0
Also, remove checks for null call context during call time, it can never be null after googleapis/gax-java#600.
And improve construction time null checks